1. Field of the Invention
The present invention relates to an information disclosing apparatus for disclosing data owned by individuals to third parties, and a multi-modal information input/output system.
2. Description of the Related Art
(1) The environments of networks in each university and each business enterprise and networks connecting universities, business enterprises, or homes have been recently prepared. Along with the cost reduction in information communication equipments and memory devices, each individual can actively or passively receive to have a large amount of information. However, exchange of the large amount of information causes a flood with information. It becomes difficult for an end user to collect necessary information from wide network environments due to the limited capacity and time of the end user. Even a user who is familiar to a network to some extent is difficult to find information of a profession different from his own profession. Each end user relies on another user who may have necessary information or another user who may know the source of the information.
In environments where individual persons have a wealth of sources and are active, access to a user having an information collection and filtering capacity is concentrated. Even the user having such a capacity cannot take time only to provide information. It takes time to understand the contents of each individual question and provide a proper answer. Smooth information exchange is interfered by delays, omissions, and misunderstanding because such exchange is performed by a communication means on a network, thus further increasing the wasteful time.
In order to solve the above problem, there is provided a method in which information owned by each user is disclosed and searched by a necessary user as needed. It is difficult and takes time to exhibit an information structure which is accessible to any user. Information constantly increases and changes, and the information structure itself must be changed accordingly. To assure security and perform smooth communications, the range of information to be disclosed and its exhibition method must change depending on partners. These management operations are not enjoyable in proportion to the required time. Even a user familiar to data collection tends to stay away from these time-consuming operations. For this reason, most of users who have a large amount of information forget the locations of the information, and such information becomes dead information. A user who wants to find necessary information must find it from all pieces of information of another user who may have this information. When the information structure is complicated, the retrieval operation becomes time-consuming. The problem is posed even in use of an existing database if the user is not familiar to the retrieval function of this database. As a result, the filing capacity of each individual is not utilized.
In order to solve this problem, an agent function is proposed in which necessary information is collected from a network in response to an instruction given by a user. When the user does not have knowledge of information retrieval environments, an accurate instruction cannot be given. When a time lag is present from the input of an instruction to the end of retrieval or the report of a result, the deficiency of an instruction causes repetition of the trial and error, resulting in wasting the time. To solve this problem, it is difficult to realize an advanced instruction analysis function capable of causing an end user to simply input a complicated instruction, or to impart a wealth of knowledge for network environments to an agent.
If a retrieval destination is individual information, a time lag tends to occur because a permission must be obtained from an owner unless necessary information is obvious disclosed information. A user who discloses information is required for extra operations if the range of contents to be disclosed varies depending on different partners. Even a user who obtains or makes a permission may hate watching information of others or hate others to watch his own information. The pieces of information owned by each individual have different formats and structures. It is very difficult for an agent to notify all these different formats and structures. It is difficult to force to standardize these pieces of information because they are owned by different individuals.
An attempt has been made to virtually reproduce working place environments (virtual office) in a computer, a worker located in the working place is expressed as an agent, and simple question-answer processing is performed in accordance with a response from the agent even in the absence of a worker (Jpn. Pat. Appln. KOKAI Publication No. 6-95997). If the type of task is limited to a specific field, a multi-modal interface system sample capable of performing a sales operation in place of a salesperson is manufactured (e.g., Jpn. Pat. Appln. KOKAI Publication No. 5-216618). Such a system, however, has the outer appearance designed by a system designer and outputs responses based on predetermined knowledge expressions and response rules. The system is not developed to an end user so as to disclose individual information and knowledge of his own in an arbitrary form. If a system does not have a certain degree of freedom, the expressions of the user who discloses information cannot be satisfied. At the same time, a user who wants to obtain information cannot be interested in standardized answers. Therefore, this system may be used for only regular business operation information.
(2) Since computers have recently been popular, automation using computers has spread over in a variety of fields. At the same time, a natural interface is implemented so as to allow an end user to use a computer. In particular, along with the improvement of recognition techniques, research and development have been made for a multi-modal interface system capable of understanding the intention of a user from the natural language uttered by the user or an image or speech and responding to the user. A smooth interactive operation between a machine and a user is impossible without using various behaviors of users.
As in a video camera for a cash dispenser, the response history between users and the machine may often be recorded to assure the security and manage the system. In an information retrieval system involving charges and copyrights, a history representing a correspondence between specific information and a specific user who is provided with the specific information must be positively recorded.
A user who uses such a system worries about easy recording of a response history because this may infringe privacy and the right of likeness. A possibility of misuse of the response history is normally regarded to be low because the social pressure acts on an enterprise which manages the system. In a public message exchange system such as a message dial, a message registered once cannot be deleted, but a message can be checked and reentered prior to its registration. When a predetermined period of time has elapsed, the record is erased.
Along with the development of multi-modal interface techniques, it is possible for an end user to offer an information service through such a system. In this case, the management of a response history depends on the intention of the user who offers the service. A possibility of misuse of this response history becomes high because of a low social pressure acting on the end user and careless management. That is, a user who enjoys the service may not stand in front of a video camera or microphone with a sense of security.
(3) Along with the preparation of network environments, information collected by individuals tends to be actively shared by a plurality of users. Operations such as formation of an agent function on this network, disclosure of necessary individual information, or collection of other individual information have been considered.
Pieces of individual information disclosed to many and unspecified persons are classified into information which can be disclosed to users and information which cannot be disclosed to them. In addition, the disclosure enable range changes. That is, agent operations must change depending on different users.
The agent operations change depending on different users in the following attempt. The priority is determined in advance in accordance with the ranks, posts, and question contents of users, and the agent operations are decided in accordance with this priority. The reliability of a user for an agent (finally the reliability of a user for an information provider) is actually updated in every conversation. An agent is desired to take proper steps to meet the instantaneously changing statuses of users without being restricted to the rules. This also applies to Jpn. Pat. Appln. KOKAI Publication No. 6-95997 and Jpn. Pat. Appln. KOKOKU Publication No. 6-79304 in which the current status of an information provider is analyzed by an explicit user's demand to directly connect a line.
When an agent cannot meet the demand of a user, the right to answer is transferred to the information provider. It is easily anticipated that a line is directly connected to the information provider every time the agent cannot answer a question. In this case, the operation efficiency of the information provider is degraded, and an effect resulting from the use of an agent cannot be obtained.
In addition, the arrangement of acquired information in an optimal category, and setup of an appropriate disclosure range are time-consuming jobs to the information provider.
(4) Along with the development of database techniques and communication network techniques, a system having a variety of functions and forms has recently been proposed as a system for retrieving information demanded by a user and providing it to the user. For example, a database service retrieves information covered by a subscription to a user as a subscriber in accordance with a demand falling within a predetermined range and providing the retrieved information to the user. A CAI system provides an answer or teaching to a student's question so as to match the degree of familiarity of the student. A claim processing system and a guide system to a customer receives a demand in a form specified in each business operation and provides information. Jpn. Pat. Appln. KOKAI Publication No. 6-95997 proposes a method for automating an information providing job to the third party using an agent serving as a human agency business system.
These various types of information providing systems are effective in the corresponding services and business operations, and their forms are limited to provide predetermined specific data to specific users. For example, in a most advanced CAI system, information matching the degree of familiarity of students and the progress of curricula of students is effectively provided on the basis of the user model of students. The range of this most advanced CAI system aims at education in a specific field and is limited only to provide information to specific users as students.
At present as well as in the future, most of pieces of electronic information stored in a computer are data owned by individual persons at individual sites. Exchange of these pieces of information is performed through electronic mails, news systems, and the like. Each individual person manually acquires or provides information. The acquisition of information depends on knowledge associated with individual knowledge associated with specific locations and persons as information sources (e.g., human relationships and networks). Information is provided by only the voluntary activity of each individual person, e.g., by an answer to a question through an electronic mail. Under these circumstances, data owned by each individual person cannot be effectively utilized and are possessed among persons having a certain human relationship. The voluntary information exchange activity of each individual person is limited. The time and labor required to answer to a question through, e.g., an electronic mail cannot be neglected. Therefore, the information disclosing jobs performed by individual persons must be automated and supported.
Information has an indefinite form and makes it difficult to determine meanings and forms in advance. Users who demand to provide information personally are not classified as, e.g., customers in the database service and students in the CAI system. For this reason, the information disclosing system for regular business operations, as described above, cannot perform human agency business for flexible and careful information exchange performed between users. That is, a method of changing a method of providing information depending on different users and a means for setting conditions for changes in information providing method have not been established.
Conditions for disclosing information generally change over time, and no countermeasure is not taken for this problem. A means for correcting a system failure resulting from the inappropriate disclosure of information has not been realized, either.
Various problems are posed to realize an information disclosing apparatus for effectively and safely disclosing useful information, owned by individual persons, in place of users.
Data owned by individual persons have various meanings, forms, confidentialities, and the degrees of difficulty. Application programs for retrieving and presenting these data are not standardized. To arrange these data for disclosure is difficult and requires much labor.
Various types of persons are assumed as information demanders who demand information to information providers. It is difficult with much labor to assume all responses to the demands from the various types of persons. In particular, whether private information of an information provider is disclosed or how information within a specific range is disclosed depends on various factors such as the personal relationship between the information provider and an information demander, ages and sexes of the users, and organizations to which the users belong.
The confidentiality and the degree of difficulty of data owned by an information provider as well as the right and the degree of familiarity of an information demander dynamically change over time. Even if the method and limitations for disclosing a variety of information to a variety of persons are set in detail, these may become invalid with a lapse of time. Even if the information disclosing apparatus fails to provide information which can meet a user's demand, it is desired not to repeat the failure in the same status.
As has been described above, a conventional information disclosing apparatus has the following problems:
(1) A large amount of information collected by individual persons cannot be easily disclosed in a conventional information disclosing apparatus. There is no means for adjusting the degree of disclosure from an information owner to specific types of persons.
(2) In a conventional information disclosing apparatus, a response history is managed by a person who owns the information disclosing apparatus. For this reason, a user who feels doubt to the manager of the response history cannot use this information disclosing apparatus.
(3) The conventional information disclosing apparatus does not have a means for dynamically reflecting the status of a user and its change in a negotiation step with an agent and disclosing information so as to take proper steps to meet instantaneously changing status of the user.
There is no means for grasping the status of an information provider and considering an effective countermeasure to the information provider.
There are neither a means for arranging acquired information in a proper category nor a means for reflecting a negotiation process and other negotiations and flexibly correcting the disclosure range.
(4) The conventional information disclosing apparatus has no means for providing countermeasures for a personal relationship between an information provider and an information demander which dynamically changes with time or for the confidentiality and the degree of difficulty of data owned by the information provider. Therefore, it is difficult to constantly disclose information in an easy and appropriate form.
In recent years, human interfaces employing a plurality of information transmission means possessed by men, such as a natural language interface/multi-modal interface, have been enthusiastically created.
In addition to a variety of media, transmission of emotions between speakers plays an important role as a factor for efficiently progressing interactive operations. In practice, the interactive operations do not often reach a conclusion due to differences in contextual recognition and interpretation of utterance intentions between speakers. These circumstances necessitate an apparatus for supporting extraction/recognition/transmission of the psychological status or emotions of a speaker.
As for recognition and understanding, and synthesis of the intention and emotional information of speech, a variety of studies have been reported, such as Kanazawa et. al. (Journal of the Electronics, Information, and Communication Engineers of Japan D-11, Vol. J77-D-11, No. 8, pp. 1,512-1,521) and Cahn et. al. ("Generating Expression in Synthesized Speech", Technical Report, Masachusett Institute of Technology, 1990).
These are based on "rhythmic information" such as the pitch and accents of speech signals, and intention information and emotional information such as anger, joy, sorrow, agreement, admiration, and connection are recognized/understood and generated.
An attempt has been made to recognize emotions from a character string expressing emotions in a text. Examples are Fujimoto et. al. proposing a scheme (Jpn. Pat. Appln. KOKOKU Publication No. 6-822376) in which emotional information included in a word string is extracted using an emotion dictionary registered upon numerical conversion of emotional information in units of words and a scheme (Jpn. Pat. Appln. KOKOKU Publication No. 6-822377) having rules for converting the emotion information on the basis of a syntactic analysis result.
The above schemes extract emotions included in one sentence or utterance, but do not recognize the emotions of a user throughout the interactive operation. In practice, the emotions of users often vary depending on situations or statuses even if the same utterance or verbal expression is used. In addition, utterances or verbal expressions for expressing the emotions vary depending on different users.
An attempt has been made to recognize the emotions of main characters in accordance with the narration of a story. Studies by W. G. Lehnert et. al. ("The Role of Affect in Narrative Structure", Cognition and Emotion, 1987, pp. 299-322) and M. G. Dyer ("Emotions and their Computations: Three Computer Models", 1987, Lawrence Erlbaum Associates Limited) have been reported. The situation is recognized in accordance with sentences described in a natural language to reason the emotional states of the main characters. The verbal expression and situation which represent emotions are extracted from the narration of the story. However, the emotions and situation using only the contents of utterances and conversations of the main characters is not recognized.
Interactive systems having chat structure models to generate appropriate responses for the utterance intentions of users have also been extensively studied. Examples of interactive operations of texts are proposed by Sumida et. al. ("Consideration of Naturalness of Responses in Question-Answer System", Shigaku Giho NLC86-16, pp. 25-32, 1986) and Ukita et. al. ("Equipment Manipulation Guide System by Natural Language Input", Shigaku Giho OS-88-18, pp. 13-18, 1988). An example of interactive operations of speech is proposed by Araki et. al. ("Understanding of Utterances Using Structure of Interactive Operations and Concept of Words", Joho Shori Gakkai, 42nd National Meeting, 3, pp. 61-62, 1991). These prior arts aim at recognizing a user's intention corresponding to the situation from a user's utterance or input text and generating an appropriate response. However, they do not recognize the emotions of the user.
In an information disclosing apparatus and a multi-modal information input/output system described in Japanese Patent Application No. 7-86266, emotion recognition is decided using emotion information included in one sentence or utterance. Information associated with the status of an interactive operation is not utilized.
As described above, no conventional information disclosing apparatus has a means for analyzing the emotions of a user on the basis of the status of the interactive operation. For this reason, it becomes difficult to generate a response in consideration of the emotions of the user. Misunderstanding occurs in the interactive operations to erroneously understand the intention of the user. In this manner, an unnecessary mental load is imposed on the user.